Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization

In this paper, we study multi-block min-max bilevel optimization problems, where the upper level is non-convex strongly-concave minimax objective and the lower level is a strongly convex objective, and there are multiple blocks of dual variables and lower level problems. Due to the intertwined multi-block min-max bilevel structure, the computational cost at each iteration could be prohibitively high, especially with a large number of blocks. To tackle this challenge, we present a single-loop randomized stochastic algorithm, which requires updates for only a constant number of blocks at each iteration. Under some mild assumptions on the problem, we establish its sample complexity of $O(1/\epsilon^4)$ for finding an $\epsilon$-stationary point. This matches the optimal complexity for solving stochastic nonconvex optimization under a general unbiased stochastic oracle model. Moreover, we provide two applications of the proposed method in multi-task deep AUC (area under ROC curve) maximization and multi-task deep partial AUC maximization. Experimental results validate our theory and demonstrate the effectiveness of our method on problems with hundreds of tasks

Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization

In this paper, we consider non-convex multi-block bilevel optimization (MBBO) problems, which involve $m\gg 1$ lower level problems and have important applications in machine learning. Designing a stochastic gradient and controlling its variance is more intricate due to the hierarchical sampling of blocks and data and the unique challenge of estimating hyper-gradient. We aim to achieve three nice properties for our algorithm: (a) matching the state-of-the-art complexity of standard BO problems with a single block; (b) achieving parallel speedup by sampling $I$ blocks and sampling $B$ samples for each sampled block per-iteration; (c) avoiding the computation of the inverse of a high-dimensional Hessian matrix estimator. However, it is non-trivial to achieve all of these by observing that existing works only achieve one or two of these properties. To address the involved challenges for achieving (a, b, c), we propose two stochastic algorithms by using advanced blockwise variance-reduction techniques for tracking the Hessian matrices (for low-dimensional problems) or the Hessian-vector products (for high-dimensional problems), and prove an iteration complexity of $O(\frac{m\epsilon^{-3}\mathbb{I}(I<m)}{I\sqrt{I}} + \frac{m\epsilon^{-3}}{I\sqrt{B}})$ for finding an $\epsilon$-stationary point under appropriate conditions. We also conduct experiments to verify the effectiveness of the proposed algorithms comparing with existing MBBO algorithms

Long-term post-traumatic stress symptoms in COVID-19 survivors and its risk factors: a two-year longitudinal cohort study

The COVID-19 pandemic has led to widespread mental health problems, necessitating the investigation of lon-gitudinal mental health changes, associated risk factors, and neural mechanisms in survivors. We recorded de-mographics, mental health, social support, and potential exposures in survivors at 3 months (n = 189), 6 months (n = 47), and 2 years (n = 69) post-discharge and collected brain imaging data at the second timepoint. Control groups included non-COVID-19 locals (3 months: n = 188, 6 months: n = 42, 2 years: n = 71). Results indicated that female survivors exhibited higher post-traumatic stress symptoms (PTSS) and depression levels than female controls for up to 2 years, along with higher anxiety level for up to 6 months. Male survivors had higher PTSS, depression, and anxiety levels than male controls at 2 months. Moreover, COVID-related trauma and low social support were risk factors for PTSS and negative emotions in survivors. Neuroimaging revealed increased amygdala activity in male survivors and correlations between hippocampus activity and depression symptoms as well as between right hippocampus activity and social support. Our study emphasized the importance of monitoring mental wellness in COVID-19 survivors and underscored the crucial role of social support in miti-gating mental health problems

Transcriptome and Network Analyses Reveal the Gene Set Involved in PST Accumulation and Responses to Toxic Alexandrium minutum Exposure in the Gills of Chlamys farreri

Bivalve molluscs are filter-feeding organisms that can accumulate paralytic shellfish toxins (PST) through ingesting toxic marine dinoflagellates. While the effects of PST accumulation upon the physiology of bivalves have been documented, the underlying molecular mechanism remains poorly understood. In this study, transcriptomic analysis was performed in the gills of Zhikong scallop (Chlamys farreri) after 1, 3, 5, 10, and 15 day(s) exposure of PST-producing dinoflagellate Alexandrium minutum. Higher numbers of differentially expressed genes (DEGs) were detected at day 1 (1538) and day 15 (989) than that at day 3 (77), day 5 (82), and day 10 (80) after exposure, and most of the DEGs were only regulated at day 1 or day 15, highlighting different response mechanisms of scallop to PST-producing dinoflagellate at different stages of exposure. Functional enrichment results suggested that PST exposure induced the alterations of nervous system development processes and the activation of xenobiotic metabolism and substance transport processes at the acute and chronic stages of exposure, respectively, while the immune functions were inhibited by PST and might ultimately cause the activation of apoptosis. Furthermore, a weighted gene co-expression network was constructed, and ten responsive modules for toxic algae exposure were identified, among which the yellow module was found to be significantly correlated with PST content. Most of the hub genes in the yellow module were annotated as solute carriers (SLCs) with eight being OCTN1s, implying their dominant roles in regulating PST accumulation in scallop gills. Overall, our results reveal the gene set responding to and involved in PST accumulation in scallop gills, which will deepen our understanding of the molecular mechanism of bivalve resistance to PST

Statistical properties of kinetic- scale magnetic holes in terrestrial space

Kinetic- scale magnetic holes (KSMHs) are structures characterized by a significant magnetic depression with a length scale on the order of the proton gyroradius. These structures have been investigated in recent studies in near- Earth space, and found to be closely related to energy conversion and particle acceleration, wave- particle interactions, magnetic reconnection, and turbulence at the kinetic- scale. However, there are still several major issues of the KSMHs that need further study - including (a) the source of these structures (locally generated in near- Earth space, or carried by the solar wind), (b) the environmental conditions leading to their generation, and (c) their spatio- temporal characteristics. In this study, KSMHs in near- Earth space are investigated statistically using data from the Magnetospheric Multiscale mission. Approximately 200,000 events were observed from September 2015 to March 2020. Occurrence rates of such structures in the solar wind, magnetosheath, and magnetotail were obtained. We find that KSMHs occur in the magnetosheath at rates far above their occurrence in the solar wind. This indicates that most of the structures are generated locally in the magnetosheath, rather than advected with the solar wind. Moreover, KSMHs occur in the downstream region of the quasi- parallel shock at rates significantly higher than in the downstream region of the quasi- perpendicular shock, indicating a relationship with the turbulent plasma environment. Close to the magnetopause, we find that the depths of KSMHs decrease as their temporal- scale increases. We also find that the spatial- scales of the KSMHs near the subsolar magnetosheath are smaller than those in the flanks. Furthermore, their global distribution shows a significant dawn- dusk asymmetry (duskside dominating) in the magnetotail.Key PointsMost KSMHs are locally generated in the magnetosheath, rather than advected with the solar wind.KSMHs are more likely to be generated downstream of the quasi- parallel shock, indicating the importance of turbulence in their generation.The scale- size of KSMHs is smaller near the subsolar magnetosheath than along the flanks, indicating they may be affected by the magnetosheath pressure environment.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/167119/1/epp320195.pd